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ABSTRACT 

This paper discusses the development and results of a new tool used for measuring shifts in students' perceptions of earth and 
ocean sciences called the Student Perceptions about Earth Sciences Survey (SPESS). The survey measures where students lie 
on the novice-expert continuum, and how their perceptions change after taking one or more earth and ocean science 
course(s). The survey is composed of 29 statements that have been validated with expert responses and student interviews. 
Factor analysis was used to group statements into seven categories, which provide instructors with valuable information to 
guide pedagogical change in their classes. Overall student perceptions in earth and ocean sciences show little change over the 
course of the term, in contrast to novice shifts in perceptions seen in other science disciplines. Students in earth and ocean 
sciences also do not display any gender differences, unlike those seen in these other disciplines. However, differences are seen 
between face-to-face and distance education versions of a course, between earth and ocean science majors and nonmajors, 
and between a community college and a large research institution. The former of each pair are more expert-like. © 2012 
National Association of Geoscience Teachers. [DOI: 10.5408/10-199.1] 
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INTRODUCTION 

One of the main goals of an undergraduate education is 
to move students from novice-like thinking towards expert¬ 
like thinking. Not only does this mean gaining knowledge of 
facts and concepts, but also changing perceptions of the 
disciplines studied in terms of how they are used and 
learned. In earth and ocean science, the Geoscience Concept 
Inventory (GCI) is one example of an instrument that can be 
used to measure the former (Libarkin and Anderson, 2005). 
While some surveys have been used to measure student 
perceptions towards earth and ocean science in grade school 
(e.g., Mao and Chang, 1998), no validated survey has been 
developed to measure general perceptions towards the 
discipline at the postsecondary level. 

Here we report on the development of the Student 
Perceptions about Earth Science Survey (SPESS) to measure 
student perceptions (Appendix A). The SPESS is based on a 
perception survey developed by researchers in physics 
education, the Colorado Learning Attitudes about Science 
Survey (CLASS), which is appropriate for majors and 
nonmajors across all subdisciplines of physics (Adams et 
ah, 2006). The CLASS is now widely used internationally, 
and has since been adapted for the Chemistry, Biology and 
Geoscience Departments at the University of Colorado 
(Barbera et al., 2008; W. Adams, pers. comm.). 
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The main goals devised for the development of the SPESS 
are to (1) establish students' beliefs about the nature and 
relevance of earth and ocean science, (2) determine whether 
students perceive earth and ocean sciences the same way that 
experts in the field do, and (3) reveal the effects of course 
innovations on student perceptions and interest. 

Previous surveys have been developed in order to 
measure student perceptions toward science in general 
(e.g., Fennema and Sherman, 1976; Aikenhead and Ryan, 
1992; Libarkin, 2001; Lederman et al., 2002); however, they 
have different goals than the SPESS. The statements that are 
found on the SPESS and CLASS are specific to a discipline 
(earth and ocean science or physics). When these statements 
are phrased as pertaining to science rather than the 
discipline, students do not know how to answer (Adams et 
al. 2006). For example, when asked if they agree or disagree 
with "Understanding science basically means being able to 
recall something you've read or been shown," students 
reply, "It depends on whether you mean biology or physics." 
Thus, views of science in general are not specifically 
addressed in the SPESS; just the perceptions and beliefs 
towards the broad field of earth and ocean science are the 
focus of this survey. 

The above goals have been used to guide the overlying 
research questions for the survey. Namely, what are 
students' perceptions about the nature and relevance of 
earth and ocean science in relation to those of an expert? Do 
these perceptions change over the course of a term, and if so, 
are there different groups of people (e.g., males vs. females) 
or different instructional styles (e.g., face-to-face vs. distance 
education) that show different results? 

The source of students' personal beliefs is typically their 
experiences in schools prior to postsecondary and experience 
in their out-of-school lives (Gal et al., 1997). The nature of 
these classroom experiences (e.g., style/quality of instruc¬ 
tion) can help predict student performance alongside 
previous math and science scores (Tai et al., 2006). Student 
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beliefs about the nature of a discipline impact (1) "process 
considerations," or how students approach learning, (2) 
"outcome considerations," or how they translate their 
understanding to the world outside of the classroom, and 
(3) "access considerations," whether or not they choose to 
continue on in their studies of a discipline (Gal et al., 1997). 
Perceptions correlate with what courses a student chooses to 
take, and students' conceptual learning gains in these 
courses (Perkins et al., 2005). Thus, a general goal of 
education is to have students come to perceive the discipline 
more like an expert in the field, which in turn should lead to 
improved learning gains. 

Experts not only have greater conceptual knowledge of 
the discipline, but they also hold very different views about 
how new knowledge is gained, structured, and manipulated. 
Hammer (1994) reviewed the literature that discusses the 
differences between novice and expert beliefs about science in 
the context of physics with regards to: (1) the structure of 
knowledge, (2) the content of knowledge, and (3) the source 
of knowledge. First, experts see knowledge within a discipline 
as comprising one coherent unit, and can usually rederive 
things if forgotten, or solve problems with a different 
approach (often many). Novices do not see this coherence, 
and think of the discipline as a grouping of separate pieces. 
They believe that to know these pieces is simply to remember 
them (Chi et al., 1981). Second, experts think that knowledge 
is made up of concepts that can be represented by equations, 
much like another language. They find this language to be 
fully translatable, and often common sense-like. Problems are 
solved by conceptualization. Novices think that knowledge is 
made up of facts and equations, which are not a represen¬ 
tation of concepts or words. Problem solving is guided by 
searching for the correct equation, and then computing the 
answer mathematically (Larkin, 1983). Finally, experts believe 
that the acquisition of knowledge is an independent process 
(which is possible without an instructor), and making sense of 
the ideas is extremely important. They want to understand 
these ideas, and not just memorize them. Knowledge 
represents the real world. Novices believe that knowledge is 
handed down from positions of authority and learning is only 
remembering what they have been told. Knowledge is 
isolated and the representation of the real world is not 
apparent (Kitchener and King, 1981). While the SPESS does 
not measure the above categories directly, statements were 
created based upon these common characteristics of novice 
thinkers. 

Many of the statements in the CLASS are not fully 
translatable to earth and ocean science because of the 
differences in the disciplines themselves. We have adapted 
many of the CLASS statements and developed new 
statements to produce a tool that probes the perceptions 
that are important to experts in the field of earth and ocean 
science. Similar to the CLASS, we designed the SPESS to 
work with all subdisciplines in the field of earth and ocean 
science (geology, geophysics, oceanography, and atmo¬ 
spheric science), and be accessible to people with varying 
degrees of knowledge toward the discipline, such as majors 
and nonmajors. 

Results from the SPESS are being used to track how 
course and curriculum changes affect student perceptions. 
Pretest data can provide a snapshot of the student 
population at the beginning of the semester, so pedagogy 
and curriculum can be targeted towards preheld student 


perceptions. The SPESS can then be used to measure if these 
course and program changes positively affect student 
perceptions towards the discipline. Finally, specific cohorts 
of students can be tracked through a program to analyze 
how, when, and why their perceptions change. 

SURVEY DEVELOPMENT 

Design 

Faculty interviews and feedback were used to decide 
which statements from previous attitude/perception surveys 
were relevant to earth and ocean science and to identify 
additional perceptions important to earth and ocean science. 
Ten of the 29 statements in SPESS come directly from the 
CLASS-Phys (Adams et al., 2006) with the word "physics" 
replaced by "earth and ocean science." Many of the CLASS- 
Phys statements deal with the use of mathematical 
equations, which faculty deemed were important to all the 
subdisciplines in earth and ocean science, although the use 
of equations varies across courses. Six come from the 
unpublished CLASS-Geo with the word "geoscience" 
replaced by "earth and ocean science" (J. Stempien, pers. 
comm.). Thirteen new statements were created based on 
faculty feedback. Previous versions of the survey were 
reworked by student interviews and expert validation to 
create the current version. Each statement has been 
validated to be concise, have one consistent interpretation, 
and be meaningful for even those with little to no 
knowledge of earth and ocean science. 

Early Versions 

The set of statements in the current survey is the product 
of multiple iterations starting with 46 statements. Initial 
interviews (as described below) with 14 novice students were 
used to identify ambiguous or confusing statements, or to 
clarify the wording in particular statements. Nine statements 
were completely discarded leaving 37 statements, some of 
which were slightly reworded. 

The remaining 37 statements were then used to survey 
28 experts (staff members and graduate students). These 
survey results were used to discard four additional state¬ 
ments where there was no clear expert agreement. Follow up 
interviews with experts and a realignment of the statements 
with the goals of the survey resulted in statements that were 
then revalidated below. 

Student Interviews: Current Version 

Eighteen students were interviewed to validate the 
current version of the survey. We selected six students that 
had never taken an earth and ocean science course before, 
six students that had taken one or more lower level earth 
and ocean science courses, but were majoring in another 
science discipline (applied science or arts), and six students 
that were taking higher level courses and majoring in the 
field of earth and ocean science. Lower level courses don't 
have pre-requisites and are taken either by students from 
other disciplines as electives to gain breadth or by majors as 
their introductory course. Higher-level courses are taken 
during the third and fourth years by students who are 
majoring in the discipline. This broad range of interviewees 
was obtained to test that the survey is accessible to all levels 
of students. 
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The interviewees were paid student volunteers. Each 
student was initially asked some demographic information 
(age, major, favorite classes, career goal, etc.), both to ensure 
that the diverse population of the University of British 
Columbia (UBC) was represented and to make them more 
comfortable with the interview experience. Then, the student 
took the survey by pencil and paper. Finally, the survey was 
retaken verbally, with no access to the previous written 
answers. As the student answered the questions verbally, 
they were asked to explain their rationale to the interviewer. 
Most provided their rationale freely (without the interviewer 
having to ask again), and seemed comfortable with the 
situation. If they asked any questions of the interviewer, they 
were not answered until after the interview. Students were 
also asked to interpret the term "earth and ocean science" to 
ensure that it was consistent. 

The interviews were audio recorded and transcribed 
shortly afterward. The transcribed interviews were compared 
with the initial pencil and paper test, to ensure that they 
aligned. It was found that 21% of the responses changed to a 
different one of the three groups of responses (disagree: 1 or 
2, neutral: 3, and agree: 4 or 5). Of the students that changed 
their response, 83% were either to or from a neutral 
response, which aren't used in calculating the student's 
score on the survey (see the "Scoring" section below). Both 
the low percentage of changed responses and large number 
that involved neutral responses suggests that the consistency 
of the survey is sufficient. If the neutral responses are 
discarded, the reproducibility of the written answers is 96%. 

Listening to a verbal rationale from each student made it 
possible to determine whether their interpretation was 
consistent with the original intent of each question, and 
whether all students were interpreting the question in the 
same way. Those statements that were not interpreted in a 
way that was either consistent with the original intention 
and/or consistent between all students were either reworded 
or discarded. 

One of the major changes that occurred over the course 
of the interviews was the use of the term "earth science." 
Many students felt that earth science only includes geology, 
and not geophysics, oceanography, or atmospheric science. 
In addition, simply using the word "science" elicited very 
different responses because most of the students were not 
limiting their thoughts to earth and ocean science, and were 
extending them to physics, chemistry, biology, mathematics, 
and so on. The term "earth and ocean science" was selected, 
as interviews showed that students consistently interpreted 
this term as a representation of the discipline. It should be 
noted that the validation of the term "earth and ocean 
science" only occurred at UBC in the Department of Earth 
and Ocean Sciences, which encompasses geology, geophys¬ 
ics, oceanography, and atmospheric science. However, we 
expect that the use of this term would still be successful in 
other contexts because few undergraduates, particularly at 
the lower levels (nonmajors) used in many of the interviews, 
have much sense of the subdisciplines mentioned above. 

Half of the students were asked how they felt an expert 
in the field of earth and ocean science would respond on a 
select number of statements. All of the students interviewed 
about the expert response were completely aware of what 
the expert response was when asked, yet they always stated 
what they felt, which was often different than what they 
thought the expert would say. 
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Expert Responses: Current Version 

Experts in our department (those holding a PhD in any 
subdiscipline of earth and ocean science) took the survey 
twice over the course of the validation, once for Version 1.5 
(February 2008, n = 28) and once for Version 3.5 (July 2009, 
n = 10). From Version 1.5 to Version 3.5, 10 statements 
remained the same, 4 statements were reworded, 20 new 
statements were created, and 22 statements were removed. 

Experts were chosen to represent the breadth of our 
department, including geologists, geophysicists, oceanogra¬ 
phers, and atmospheric scientists. These results defined the 
expert response used when scoring the survey. All questions 
that did not have a consistent expert response were removed 
from the survey. 

Evidence of Validity 

Using the above student interviews and expert respons¬ 
es, the SPESS has been validated to be interpreted 
consistently by all levels of students, to be accessible to all 
subdisciplines of earth and ocean science, and to distinguish 
between general populations who are expected to be placed 
differently on the novice-expert continuum (e.g., majors vs. 
nonmajors). Additionally, results from factor analysis, 
described later, show coherent groupings of statements into 
categories, and thus, statements that are clearly associated 
with one another. 

Fall 2008 and fall 2009 responses from a large 
introductory general earth and ocean science course were 
compared for test-retest analysis. This compares survey 
responses between two administrations with similar incom¬ 
ing populations by calculating their correlation. Thus, it is a 
measure of the repeatability of the survey. Results are as 
follows: percentage favorable (agree) = 0.98, percentage 
unfavorable (disagree) = 0.99, and percentage neutral = 
0.89. The high test-retest (correlation) values between fall 
2008 and fall 2009 mean that responses given by different 
students from term to term are nearly identical, and the 
survey produces repeatable results with a consistent 
distribution. We have chosen to report the test-retest 
reliability because it supplies a measure of stability. 
Cronbach's alpha coefficient is often stated as evidence of 
reliability, but it actually measures internal consistency 
rather than stability. Internal consistency is a valuable 
measure for single construct assessments to help determine 
if in fact all the questions/statements are measuring the same 
construct. Since the SPESS is a multiple construct instru¬ 
ment, the Cronbach's alpha does not apply. 

SPESS in its current version (V4.0) contains 29 Likert 
scale statements (5 point scale, strongly disagree to strongly 
agree; Likert, 1932), as well as both a Likert scale and free 
response question on personal interest. 

ADMINISTRATION 

The SPESS is completed online outside of class time 
once during the first two weeks of the term and again during 
the last two weeks of term. It takes the students 
approximately 10 min to complete in a thoughtful manner. 
A bonus of 1% is added to the grades of students who 
complete both the pre- and postsurvey, which greatly 
increases the number of responses. The average class 
response rate for the most recent run of the survey was 
43.6%, ranging from 10.8% to 64.0%. Differences in 
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response rate can most likely be attributed to the amount of 
advertisement and reminders that were given by instructors. 

Since September 2007, more than 8,000 students at six 
institutions have taken the SPESS over the course of seven 
semesters. The four most recent semesters used the current 
and stable version of the survey. The mean age of students 
taking the survey is 20.5 years, with slightly more females 
(54.4%) than males (45.6%) enrolled in undergraduate 
programs at UBC (UBC Office of Planning and Institutional 
Research, 2010). In the first three semesters, classes 
participating were primarily first year courses open to both 
science and nonscience majors. In later semesters, more 
upper level courses that are specifically for earth and ocean 
science majors have been included. This allows us to track 
specific students through the program and differentiate 
between varying levels of students. Classes that have taken 
the SPESS span many subdisciplines of earth and ocean 
science including mineralogy, sedimentology, physical ge¬ 
ology, oceanography, and atmospheric science. The most 
recent run of the survey (winter 2010) included 13 
introductory classes for nonmajors (5 of them distance 
education), 5 introductory classes for majors, and 5 upper 
level classes for majors. 

SCORING 

To score the survey, we used the same method as other 
discipline-specific perception surveys have for ease of 
comparison. First, the scale is collapsed to a three point 
scale as it was found in interviews that the distinction 
between strongly agree and agree is different for each person 
due to differences in conservatism and confidence, consis¬ 
tent with results from Adams et al. (2006). Because of this 
differing interpretation, both "strongly agree" and "agree" 
and "strongly disagree" and "disagree" are collapsed into 
one point. Thus, there are three possible response groups: 
disagree, neutral, or agree. 

In the case of the SPESS, the expert response for some 
of the statements is "disagree." and for others it is "agree." 
Thus, we score the responses by determining the percentage 
agreement and disagreement with the expert response for 
each statement, referred to as the percentage favorable and 
unfavorable. For example, if a class has a 65% favorable 
score in a category, this means that on average they have a 
65% agreement with the expert (which could mean either 
disagreeing or agreeing with that particular statement). 
Because the selection of "neutral" occurs for variable 
reasons, the neutral responses are given no weight to the 
percentage favorable or unfavorable. 

The filter question from the CLASS is incorporated to 
catch the students that randomly choose answers and do not 
read the statements. "We use this statement to discard the 
survey of people who are not reading the statements. Please 
select 'disagree'—option 2 (not 'strongly disagree') to 
preserve your answers." Before the data are analyzed, 
students who did not select option 2 are removed from the 
data set. Duplicate entries and those who selected the same 
option for most of the statements (greater than 65%) are 
found and removed by manually searching the data. 

Typically only students with both pre- and postdata are 
analyzed because we are interested primarily in the shift of 
perceptions between before and after taking a course. 
However, at times it is of interest to look at the full 


predataset, in order to better understand the incoming 
population. 

Categorization 

Both the pre- and postsurvey results from the fall 2008 
and winter 2009 terms were analyzed using the reduced basis 
factor analysis method detailed by Adams et al. (2006). Factor 
analysis is a method commonly used in educational research 
that categorizes sets of statements by correlations apparent 
between responses to individual statements, and has been 
particularly utilized with attitudinal surveys (e.g., Gaothlo- 
bogwe et al., 2011). In both steps of the analysis, a principle 
components analysis with a direct oblimin rotation is used. 
First, an exploratory factor analysis (e.g., Schmitt and Sass, 
2011) is performed, which produces a preliminary set of 
categories using all of the statements. Typically the first set of 
categories may contain a large amount of statements that are 
not very tightly grouped together, but works towards the 
final end product. Next, a reduced basis factor analysis is 
performed with one group of statements at a time (and often 
with many varied iterations) so that the weakly related 
statements do not skew the identification of specific 
categories (see Adams et al., 2006, for more detail). 
Statements may be shifted if they correlate better with other 
categories, and categories may be split into two if it is found 
that they are stronger as independents. We did not allow for 
less than three statements to be present in a category, as this 
would weight individual statements too heavily in the 
percentage favorable for that category. In total, this analysis 
produced seven final categories that we named to best 
represent the different aspects of student thought that they 
represented. 

The strength of a category is represented by its 
robustness, calculated by an equation developed by Adams 
et al. (2006; Table I). This equation incorporates all of the 
factor analysis results that together represent statistical 
validity, including the correlation coefficients between 
statements, factor loadings, and scree plot results. The 
maximum possible value of the robustness is 10.00. 

The average of the robustness values for all the 
categories is 5.68. This value is lower than that of the 
CLASS-Phys (6.86) and the CLASS-Chem (7.30; Adams et 
al., 2006; Barbera et al., 2008). In comparison to both the 
CLASS-Phys and the CLASS-Chem, the SPESS does, 
however, contain fewer total statements, fewer statements 
in most of the categories, and no statements falling in 
multiple categories. 

The percentage favorable is calculated both overall and 
for each of the seven individual categories. The Memorization 
category characterizes the extent to which students believe 
that earth and ocean science is about memorization of 
concepts, terms, and definitions (e.g., statement 2, "Under¬ 
standing Earth and Ocean Sciences basically means being 
able to recall something you've read or been shown."). The 
Science and Society category describes how earth and ocean 
science relates to our society, in the media and in the 
government (e.g., statement 9, "Earth and Ocean Science 
predictions must be certain if we are to use them to make 
decisions that affect our society."). Mathematical Problem 
Solving includes those statements that discuss how problems 
are approached, and primarily addresses the use of 
equations and statistics in earth and ocean science (e.g., 
statement 12, "I do not expect equations to help my 
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TABLE I: The average (between pre- and postdata) robustness of the categories and the statements contained within them. 


Category Name 

Average Robustness 

Statements 

Memorization 

6.32 

2, 7, 15, 22, 24 

Science and Society 

6.24 

9, 17, 27 

Mathematical Problem Solving 

5.24 

8, 12, 19, 23 

Personal Interest 

6.25 

1, 3, 14, 21, 25 

Skeptical Reasoning 

4.66 

4, 6, 16, 26, 28 

Conceptual Problem Solving 

5.28 

11, 13, 29 

Human-Science Interaction 

5.80 

5, 10, 20 


understanding of Earth and Ocean Science ideas; they are 
just for doing calculations."). The Personal Interest category 
represents a student's curiosity towards, enjoyment of, and 
belief in the usefulness of earth and ocean science (e.g., 
statement 1, "Things that I see around me in nature often 
lead me to think about how the Earth works."). The amount 
and way that students self-check and search for information 
(e.g., discuss with other students, check the web, and 
investigate other sources) is described by the Skeptical 
Reasoning category (e.g., statement 4, "I investigate the 
source of the information on the web before I use it for an 
assignment"). The Conceptual Problem Solving category 
simply refers to whether or not students feel that they can 
make sense of the broad conceptual ideas in earth and ocean 
science, like how long landscapes take to form and how 
natural processes on earth work (e.g., statement 11, "I often 
don't really understand the underlying ideas behind how the 
Earth works"). Human-Science Interaction describes human 
impacts on the environment, making decisions in life, and 
the interpretation of earth and ocean science information 
(e.g., statement 5, "Learning Earth and Ocean Sciences 
helps me understand the impacts humans have on the 
environment."). Only three of these categories have similar 
content to those found in the CLASS-Phys (Mathematical 
Problem Solving, Personal Interest, and Conceptual Problem 
Solving). 

RESULTS AND DISCUSSION 

At UBC in each of the individual courses we typically see 
overall percentage favorable scores in the 60%-65% range, 
with little to no positive change toward the end of term. This 


is a more positive result than has been seen with the 
CLASS-Phys, -Bio, and -Chem, which found a 5 to 10% 
negative shift in their population of students (Adams et al., 
2006; W. Adams, pers. comm.). Results from the CLASS- 
Phys have shown a 5% increase at best after targeting 
specific categorical results through pedagogy (Perkins et al., 
2005). 

We find that considering the results categorically is more 
interesting than considering an overall average (Table II). 
This average tends to blur the boundaries between different 
categories, and the categories themselves serve as a clear 
way to simplify the results when picking through each 
individual statement may not be desirable. For example, 
student perceptions related to Conceptual Problem Solving 
often show a large positive change (i.e., a shift towards more 
expert-like perceptions). We find that students as a whole 
are coming in with a low self-rated understanding of the 
earth (as many of the courses are first year courses) and are 
leaving with a greater sense of understanding. They also are 
leaving with a greater appreciation of problem solving 
strategies such as the use of equations and statistics. 
Memorization commonly shows a larger decrease than any 
other category (typically in the 3%-5% range). This suggests 
that, after taking a course in the department, students seem 
to feel more strongly that learning earth and ocean science is 
about memorization. Science and Society is most often the 
category with the lowest percentage favorable (both pre- 
and postsurvey) and typically shows little to no shift in a 
term, suggesting that these societal ideas are deeply 
entrenched or change on much longer timescales. Analyzing 
the SPESS scores in this way will be useful in targeting 
particular components of curricula or instruction. 


TABLE II: Results from the winter 2009 term of an introductory general earth and ocean science class (n = 134). 


Category 

Pre % Favorable 
(Standard Error) 

Post % Favorable 
(Standard Error) 

% Shift (Standard Error) 

Overall 

60.3 (1.5) 

61.3 (1.7) 

1.0 (1.4) 

Memorization 

69.5 (2.3) 

66.4 (2.6) 

-3.1 (2.4) 

Science and Society 

29.9 (2.5) 

28.9 (2.7) 

-1.0 (2.6) 

Mathematical Problem Solving 

54.0 (2.6) 

55.6 (2.9) 

1.6 (2.5) 

Personal Interest 

70.4 (2.4) 

73.5 (2.5) 

3.0 (2.2) 

Skeptical Reasoning 

57.9 (2.5) 

57.6 (2.6) 

-0.3 (2.5) 

Conceptual Problem Solving** 

45.5 (3.0) 

57.7 (3.1) 

12.2 (3.4) 

Human-Science Interaction 

85.3 (2.2) 

82.8 (2.2) 

-2.5 (2.6) 


*p < 0.05, **p < 0.01. 
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TABLE III: Preterm results from the winter 2009 section of an upper level geology class for majors. 


Category 

% Favorable Male (n=17), SE in brackets 

% Favorable Female (n=21), SE in brackets 

Overall 

72.0 (4.0) 

62.2 (4.8) 

Memorization 

82.4 (5.4) 

64.8 (6.9) 

Science and Society 

47.1 (9.1) 

47.6 (6.7) 

Mathematical Problem Solving 

54.4 (8.4) 

54.0 (8.3) 

Personal Interest 

87.9 (5.2) 

74.1 (6.9) 

Skeptical Reasoning 

67.1 (7.3) 

54.8 (6.0) 

Conceptual Problem Solving 

68.6 (8.3) 

69.8 (6.9) 

Human-Science Interaction' 1 ' 

88.2 (4.0) 

68.3 (8.1) 


*p < 0.05, **p < 0.01. 


When the predata are sorted by gender, classes at the 
lower levels show little to no difference between males and 
females in any of the categories. This differs from what was 
observed with the use of the CLASS in physics, biology, and 
chemistry. The CLASS-Phys, -Bio, and -Chem survey results 
all showed significantly different results between genders in 
the Personal Interest category (Adams et al., 2006; W. Adams, 
pers. comm.). They all found that females scored lower 
(sometimes in excess of 30% lower) on this category (Adams 
et al., 2006). 

However, as students progress onward through the 
earth and ocean science program, differences seem to 
emerge in upper level classes (Table III). Females have less 
expert views overall, particularly in the Memorization, 
Personal Interest, Skeptical Reasoning, and Human-Science 
Interaction categories, although only the Human-Science 
Interaction category shows a statistically significant difference 
between the two. 

At this point we cannot be sure of where this difference 
originates, and why there is no difference between genders 
at the lower level. This could be attributed to students' prior 
experience with the discipline. In Canada, all students are 
introduced to physics, biology, and chemistry at the 
secondary level. However, earth science is not a required 
course, and at some schools it isn't even offered. It may be 
that students entering postsecondary and taking a course in 
earth and ocean science truly have very few preconceptions 
of the discipline, but as they continue to study it, they start to 
develop perceptions about the field of earth and ocean 


science. Further study of lower and upper level students 
including both SPESS scores and interviews would better 
inform this notion. 

Many of our lower-level courses have a distance 
education version and a face-to-face version, with some 
distance education classes being better aligned to the face- 
to-face version than others. The main reason students listed 
for taking a distance education version of a class rather than 
the face-to-face version is that the face-to-face class did not 
fit into their schedule, they have a long commute, or that it 
gives them more flexibility. At the start of term, students in 
the distance education version have a lower personal interest 
than those in the face-to-face version, and it does not 
improve much by the end of term. This may be why students 
in the distance education version chose to take a course of 
that format in the first place. 

The pre- to postsurvey shifts for distance education 
courses are often worse than for courses with an instructor in 
lecture (face-to-face; Table IV). The Memorization and 
Mathematical Problem Solving categories in particular show 
a large difference in shifts between the face-to-face and 
distance education version of the course. After taking the 
same course by distance education, students do not seem to 
be gaining the same problem solving expertise as those in 
the face-to-face version, and feel more strongly that earth 
and ocean science is about memorization. 

Earth and ocean science majors (overall) show a higher 
preterm percentage agreement with the expert than non¬ 
majors (in an introductory geology course) in all categories 


TABLE IV: Results from the winter 2010 face-to-face (FF, N = 130) and distance education (DE, N = 195) versions of an introductory 
general earth and ocean science class. 


Category 

Pre % Favorable 
(Standard Error) 

Post % Favorable 
(Standard Error) 

% Shift 

(Standard Error) 

FF 

DE 

FF 

DE 

FF 

DE 

Overall 

59.3 (1.5) 

55.9 (1.3) 

63.9 (1.5)* 

58.8 (1.3) 

4.6 (1.2) 

2.9 (1.2) 

Memorization 

67.4 (2.4) 

68.0 (2.1) 

70.8 (2.5) 

65.2 (2.1) 

3.4 (2.6) 

-2.8 (2.0) 

Science and Society 

36.3 (2.6) 

35.6 (2.3) 

38.6 (2.8) 

35.9 (2.3) 

2.3 (2.6) 

0.4 (2.4) 

Mathematical Problem Solving 

47.1 (2.8) 

48.0 (2.4) 

56.3 (2.7)* 

48.8 (2.4) 

9.2 (2.5)* 

0.9 (2.3) 

Personal Interest 

71.9 (2.5)* 

64.9 (2.1) 

75.6 (2.2)* 

67.7 (2.1) 

3.7 (2.5) 

2.8 (2.0) 

Skeptical Reasoning 

57.5 (2.3)* 

51.0 (2.0) 

58.6 (2.5) 

54.5 (2.0) 

1.1 (2.4) 

3.6 (2.2) 

Conceptual Problem Solving 

42.8 (2.9) 

37.0 (2.5) 

53.3 (3.0) 

50.3 (2.5) 

10.5 (3.3) 

13.2 (2.4) 

Human-Science Interaction 

83.3 (2.2) 

79.3 (1.8) 

87.7 (2.0) 

84.9 (1.8) 

4.4 (2.4) 

5.6 (2.2) 


*p < 0.05, **p < 0.01 (comparison of each FF value with the DE value in the column immediately to the right) 
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TABLE V: Preterm results from winter 2009 majors (overall) and nonmajors in an introductory geology course. 


Category 

% Favorable Majors (n=87), SE in brackets 

% Favorable Non-Majors (n=63), SE in brackets 

Overall** 

78.1 (1.7) 

59.2 (2.9) 

Memorization** 

84.6 (2.3) 

71.3 (3.9) 

Science and Society** 

62.8 (3.1) 

29.9 (4.0) 

Mathematical Problem Solving** 

71.7 (2.8) 

49.6 (4.1) 

Personal Interest** 

91.0 (1.4) 

65.9 (4.3) 

Skeptical Reasoning 

67.0 (3.1) 

58.1 (3.9) 

Conceptual Problem Solving** 

83.1 (2.8) 

49.5 (4.2) 

Human-Science Interaction 

82.8 (2.7) 

80.7 (3.8) 


*p < 0.05, **p < 0.01. 


TABLE VI: Preterm results from winter 2010 introductory oceanography courses: UBC and a community college (CC). 


Category 

% Favorable UBC (n=86), SE in brackets 

% Favorable CC (n=33), SE in brackets 

Overall 

56.9 (1.9) 

63.8 (2.7) 

Memorization* 

66.6 (3.1) 

79.4 (4.4) 

Science and Society** 

31.6 (3.1) 

51.5 (5.0) 

Mathematical Problem Solving 

50.3 (3.2) 

44.7 (4.3) 

Personal Interest 

62.2 (3.3) 

68.5 (4.9) 

Skeptical Reasoning 

53.8 (3.1) 

59.1 (5.4) 

Conceptual Problem Solving 

45.6 (3.6) 

53.5 (5.0) 

Human-Science Interaction 

82.0 (2.8) 

88.9 (3.1) 


*p < 0.05, **p < 0.01. 


(e.g., Mathematical Problem Solving —71.7% vs. 49.6%; Table 
V). The majors still have relatively low agreement in the 
Skeptical Reasoning category (67.0%). However, a significant 
development in source checking may occur later in a 
person's career, such as during graduate school. Tracking a 
cohort of students all of the way through the program could 
better narrow down where exactly these changes occur, and 
how consistent they are. 

Comparing similar introductory oceanography courses 
at UBC and a community college, we see more positive 
preterm responses at the community college in all but one of 
the categories (Table VI). Mathematical Problem Solving is the 
only category in which UBC students enter the class with 
more expert-like perceptions, meaning that they have a 
higher agreement with statements on equations and 
statistics and are more expert-like in their approaches to 
problem solving. The biggest differences between the 
community college students and UBC students are in the 
Memorization and Science and Society categories. Evidently 
more community college students believe that earth and 
ocean science is not so much about memorization but about 
understanding, and they are also more expert-like in the 
connections that they draw between earth and ocean science 
and the media and/or government. Involvement of multiple 
community colleges and large research institutions would 
increase the data pool for comparison, and interviews may 
help explain the differences. 

CONCLUSION 

The Student Perceptions about Earth Sciences Survey 
(SPESS) is a survey instrument designed and validated at a 


large research university to be used for determining 
students' perceptions about the discipline of earth and 
ocean science both before and after taking a course. 
Subsequently, the effect that pedagogical changes in earth 
and ocean science courses have on student perceptions can 
be determined. The SPESS builds off of the CLASS 
developed at the University of Colorado-Boulder (CU), with 
additions to encompass the needs of the field of earth and 
ocean science. 

We have used the category selection method developed 
at CU, and find it useful for learning about different aspects 
of student perceptions before and after they take a course. 
This method produced seven categories within the SPESS, 
each characterizing a different aspect of student thought. 
The percentage agreement of student with expert response is 
first calculated overall, then within the seven categories in 
order to break down student level of expertise in different 
areas of the field. 

Results show that overall, students in earth and ocean 
science at UBC enter their courses with approximately a 
60%-65% agreement with the expert, but do not become 
much more expert-like after one term has finished. More 
detailed results include: 

1. Students in earth and ocean science do not display 
the same gender differences at the lower level as 
similar surveys in physics, biology, and chemistry 
have shown, but differences do begin to emerge with 
higher-level courses. We speculate that this may be 
due to the fact that students have less exposure to 
earth science in secondary school compared to 
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traditional sciences and therefore they developed less 
preconceptions. 

2. The face-to-face version of a course often has more 
favorable shifts than its distance education counter¬ 
part. This may be due to the fact that students either 
self-select distance education classes because of lack 
of interest in the course, or they have a more positive 
or personalized experience in a face-to-face class. 

3. Students who are earth and ocean science majors 
clearly show more expert-like views than those who 
are not majors. This is what we would expect from 
majors who choose to study earth and ocean science 
as opposed to students who are taking it for breadth 
requirements. 

4. Data from an introductory oceanography course at a 
community college shows more expert-like results 
than seen with a similar course at UBC, especially 
with regards to memorization and connections 
between earth and ocean science and our society. 
There are many factors that could contribute to this 
result such as, class size, teaching philosophies, and 
maturity of students. 

The SPESS is suitable for measuring and tracking 
student perceptions about earth and ocean science. Its 
purpose is to determine what postsecondary students think 
about the discipline and to learn more about the impact of 
specific courses, pedagogies, delivery methods, or curricula 
on student shifts toward or away from expert-like percep¬ 
tions. Results can be used to inform instructional decisions 
and to monitor the impact of any changes made to courses. 
The SPESS has been completely validated, and is ready to be 
administered to earth and ocean science students at other 
institutions. 
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APPENDIX A 

Student Perceptions about Earth Sciences Survey 
(SPESS) Statements (V4.0)—University of British Co¬ 
lumbia (Vancouver) 

1. Things that I see around me in nature often lead me to 
think about how the Earth works. 

2. Understanding Earth and Ocean Sciences basically means 
being able to recall something you've read or been shown. 

3. When I look at a landscape, I sometimes try to figure out 
how it came to look that way. 

4. I investigate the source of the information on the web 
before I use it for an assignment. 
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5. Learning Earth and Ocean Sciences helps me understand 
the impacts humans have on the environment. 

6. I compare what the media says about Earth and Ocean 
Science to how I think the Earth works. 

7. In learning about the Earth, I usually memorize the 
definitions rather than make sense of the underlying ideas. 

8. If I don't remember the answer to a question on an exam, 
there's nothing much I can do (legally!) to answer the 
question. 

9. Earth and Ocean Science predictions must be certain if we 
are to use them to make decisions that affect our society. 

10. Knowing about how the Earth works is useful in making 
some decisions in life. 

11. I often don't really understand the underlying ideas 
behind how the Earth works. 

12. I do not expect equations to help my understanding of 
Earth and Ocean Science ideas; they are just for doing 
calculations. 

13. When I look at a landscape, I have an idea of how long it 
took to form. 

14. I enjoy learning Earth and Ocean Sciences. 

15. When studying Earth and Ocean Sciences, I relate the 
important information to what I already know rather than 
just memorizing it the way it is presented. 

16. To understand Earth and Ocean Sciences I discuss it with 
other students. 

17. If an Earth and Ocean finding is in the news, it means 
that it has been proven to be true. 


18. We use this question to discard the survey of people who 
are not reading the statements. Please select "disagree"— 
option 2 (not "strongly disagree") to preserve your answers. 

19. I easily get lost when Earth and Ocean Science 
explanations involve probabilities or statistics. 

20. Even when Earth and Ocean Science investigations are 
done correctly, the interpretation of the information that is 
discovered may change in the future. 

21. The subject of Earth and Ocean Science has little relation 
to what I experience in the real world. 

22. Spending a lot of time understanding why the Earth 
behaves the way it does is a waste of time. 

23. Equations used in Earth and Ocean Science are another 
way to present ideas that we can describe in words. 

24. To learn Earth and Ocean Sciences, I only need to 
memorize terms and their definitions. 

25.1 study Earth and Ocean Science to learn knowledge that 
will be useful in my life outside of school. 

26. To understand Earth and Ocean Sciences, I sometimes 
think about my personal experiences and relate them to the 
topic being analyzed. 

27. It is important for the government to approve new Earth 
and Ocean Science ideas before they can be widely accepted 
by the population. 

28. When presented in class with a controversial Earth and 
Ocean Science idea on a topic I care about, I tend to check 
what other sources say on the topic. 

29. I can usually make sense of how natural processes on 
Earth act. 


